Information processing apparatus, method for controlling the same, and storage medium storing program therefor

ABSTRACT

An information processing apparatus includes a sound information generation unit configured to generate sound information in which a physiological sound of a body is extracted from an acquired sound, a sound direction detection unit configured to detect a direction of the physiological sound of the body based on the sound information, a facial state detection unit configured to detect a state of a face existing in the direction of the physiological sound of the body detected by the sound direction detection unit, from an image captured and generated by an imaging unit, and a contamination information generation unit configured to generate contamination information in which a predetermined area according to the state of the face is set as a contaminated area, based on the state of the face detected by the facial state detection unit.

BACKGROUND Field of the Disclosure

The present disclosure relates to an information processing apparatus, amethod for controlling the information processing apparatus, and astorage medium storing a program for the control method.

Description of the Related Art

In recent years, diverse techniques for preventing the spread ofinfections have been proposed. Japanese Patent Application Laid-Open No.2013-176471 proposes a technique for capturing in the direction of asound source, measuring the body temperature of a person by using aninfrared camera or collecting information about the mask wearing stateto generate physical condition information for a group of people, andperforming processing of preventing an infection epidemic.

An infection may be spread when droplets containing pathogens arescattered by an utterance, sneeze, or cough of a person, or anotherperson touches a contaminated area with droplets. To prevent such anepidemic, it is effective to clean areas contaminated with droplets.

SUMMARY

The present disclosure has been devised in view of the above-describedsituation and is directed to identifying contaminated areas to whichdroplets scattered from persons adhere, and easily recognizing thecontaminated areas to be cleaned.

An information processing apparatus includes a sound informationgeneration unit configured to generate sound information in which aphysiological sound of a body is extracted from an acquired sound, asound direction detection unit configured to detect a direction of thephysiological sound of the body based on the sound information, a facialstate detection unit configured to detect a state of a face existing inthe direction of the physiological sound of the body detected by thesound direction detection unit, from an image captured and generated byan imaging unit, and a contamination information generation unitconfigured to generate contamination information in which apredetermined area according to the state of the face is set as acontaminated area, based on the state of the face detected by the facialstate detection unit.

Further features of various embodiments will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example configuration of an information processingapparatus according to a first exemplary embodiment.

FIG. 2 is a flowchart illustrating an example of contaminated areadetection and generation processing according to the first exemplaryembodiment.

FIG. 3 is a flowchart illustrating an example of detection processingaccording to the first exemplary embodiment.

FIG. 4 is a flowchart illustrating an example of contaminationinformation generation processing according to the first exemplaryembodiment.

FIG. 5 is a flowchart illustrating an example of contaminationinformation generation processing according to the first exemplaryembodiment.

FIG. 6 is a flowchart illustrating an example of contaminationinformation generation processing according to the first exemplaryembodiment.

FIG. 7 illustrates an example of contamination information presentationaccording to the first exemplary embodiment.

FIG. 8 illustrates an example of an information processing systemaccording to a second exemplary embodiment.

FIG. 9 is a flowchart illustrating an example of contaminationinformation presentation processing according to the second exemplaryembodiment.

FIG. 10 illustrates an example configuration of an informationprocessing apparatus according to a third exemplary embodiment.

FIG. 11 is a flowchart illustrating an example of contaminated areadetection and generation processing according to the third exemplaryembodiment.

FIGS. 12A to 12C illustrate movement information for a moving object.

FIGS. 13A to 13E illustrate update of contamination information based onthe movement information.

FIG. 14 is a flowchart illustrating an example of contaminated areadetection and generation processing according to a fourth exemplaryembodiment.

FIG. 15 illustrates an example configuration of a cleaning apparatusaccording to a fifth exemplary embodiment.

FIG. 16 is a flowchart illustrating an example of cleaning operationprocessing according to the fifth exemplary embodiment.

FIG. 17 illustrates the cleaning operation according to the fifthexemplary embodiment.

FIG. 18 illustrates an example configuration of a cleaning apparatusaccording to a sixth exemplary embodiment.

FIG. 19 is a flowchart illustrating an example of contaminated areadetection and generation processing according to the sixth exemplaryembodiment.

FIG. 20 is a flowchart illustrating an example of data acquisitionprocessing according to the sixth exemplary embodiment.

FIGS. 21A to 21D illustrate examples of stored data, priority data, andcleaning result determination data in a data acquisition unit.

FIG. 22 illustrates a cleaning operation according to the sixthexemplary embodiment.

FIG. 23 is a flowchart illustrating an example of cleaning operationprocessing according to the fifth exemplary embodiment.

FIG. 24 is a flowchart illustrating an example of cleaning resultdetermination processing according to the sixth exemplary embodiment.

FIG. 25 is a block diagram illustrating an example configuration of aninformation processing apparatus according to a seventh exemplaryembodiment.

FIG. 26 is a flowchart illustrating an example of cleaning operationprocessing according to the seventh exemplary embodiment.

FIG. 27 is a flowchart illustrating an example of cleaning operationprocessing according to the seventh exemplary embodiment.

FIG. 28 is a flowchart illustrating an example of cleaning operationprocessing according to the seventh exemplary embodiment.

FIGS. 29A to 29C illustrate examples of cleaning processing by acleaning unit according to the seventh exemplary embodiment.

DESCRIPTION OF THE EMBODIMENTS

Exemplary embodiments will now be described with reference to theaccompanying drawings.

FIG. 1 is a block diagram illustrating an example configuration of aninformation processing apparatus 101 according to a first exemplaryembodiment. The information processing apparatus 101 includes a soundacquisition unit 102, a sound information generation unit 103, a sounddirection detection unit 104, an imaging unit 105, a facial statedetection unit 106, a contamination information generation unit 107, aninformation presentation unit 108, a system control unit 109, and acommunication bus 110. The sound acquisition unit 102, the soundinformation generation unit 103, the sound direction detection unit 104,the imaging unit 105, the facial state detection unit 106, thecontamination information generation unit 107, the informationpresentation unit 108, and the system control unit 109 are communicablyconnected with each other via the communication bus 110.

The sound acquisition unit 102 includes, for example, a plurality ofmicrophones to acquire a sound.

The sound information generation unit 103 generates sound information inwhich a physiological sound of the body is extracted from the soundacquired by the sound acquisition unit 102. Examples of thephysiological sound of the body include sounds generated by actions,such as an utterance, sneeze, cough, blow, and exhalation. For example,the sound information generation unit 103 stores patterns ofphysiological sounds of the body, performs pattern matching with soundsacquired by the sound acquisition unit 102, and extracts a physiologicalsound of the body. The sound information generation unit 103 extracts aphysiological sound of the body for each microphone included in thesound acquisition unit 102. The sound information generation unit 103also generates sound volume information indicating the sound volume ofthe physiological sound of the body.

The sound direction detection unit 104 detects a sound direction basedon the sound information generated by the sound information generationunit 103. The sound direction detection unit 104 detects a direction ofthe physiological sound of the body, for example, based on the positionsof the plurality of microphones composing the sound acquisition unit 102and time differences between physiological sounds of the body extractedby the sound information generation unit 103 for each microphone

The imaging unit 105 captures a subject to generate an image of thesubject. The imaging unit 105 is, for example, an image sensor such as acharge coupled device (CCD) image sensor and a complementary metal oxidesemiconductor (CMOS) image sensor. Although, in the example illustratedin FIG. 1 , the imaging unit 105 is provided in the informationprocessing apparatus 101, an apparatus having the function of theimaging unit 105 may be installed outside the information processingapparatus 101 to provide the information processing apparatus 101 withcaptured images.

The facial state detection unit 106 detects a face existing in thedirection of the sound detected by the sound direction detection unit104 to detect the facial state based on the image generated by theimaging unit 105. Examples of the facial state include a faceorientation, a mask wearing state, and a distance between the imagingunit 105 and the face. The facial state detection unit 106 detects theface orientation based on, for example, a layout of facial featurepoints, such as eyes, nose, and mouth in the detected face. The facialstate detection unit 106 also detects a mask wearing state, for example,according to whether the nose and mouth can be detected out of facialfeature points such as the eyes, nose, and mouth in the detected face.The facial state detection unit 106 also detects, for example, adistance between the imaging unit 105 and the face depending on thedistances between facial feature points, such as the eyes, nose, andmouth, in the detected face.

The contamination information generation unit 107 generatescontamination information based on the facial state detected by thefacial state detection unit 106. The contamination information indicatesa predetermined area according to the facial state as a contaminatedarea to which scattered droplets adhere. The contamination informationincludes information that enables identifying a contaminated area, suchas a distance, direction, and shape of the contamination. The distanceof the contamination refers to the distance from the face (scatteringsource) to the area where scattered droplets reach, and the direction ofthe contamination refers to the direction in which droplets arescattered. The contamination information also includes the position of acontaminated area based on the distance between the imaging unit 105 andthe face detected by the facial state detection unit 106 and thedirection of the sound detected by the sound direction detection unit104.

If the detected face wears a mask, the contamination informationgeneration unit 107 shortens the distance of the contamination includedin the contamination information, sets the direction of thecontamination to the direction of the leakage from the gap between themask and the face, and sets the shape of the contamination to the shapeof the leakage from the gap between the mask and the face. Thecontamination information generation unit 107 also changes thecontamination information to increase the distance of the contaminationor the shape size of the contamination with increasing sound volumebased on the sound volume information generated by the sound informationgeneration unit 103.

The contamination information generation unit 107 also storescontamination information satisfying a predetermined storage conditionand combines pieces of the stored contamination information to generatecombined contamination information. The predetermined storage conditionrefers to the fact that, for example, the contamination information isgenerated in the time period when an instruction is issued from the uservia the information presentation unit 108 or the time period since thesound acquisition unit 102 starts the sound acquisition until aninstruction for presenting the combined contamination information isissued by the user. The contamination information generation unit 107also deletes the contamination information satisfying a predetermineddeletion condition out of the stored contamination information. Thepredetermined deletion condition refers to the fact that, for example,the time period when the contamination information is generated isincluded in the time period when an instruction is issued from the uservia the information presentation unit 108 or the time period since thesound acquisition unit 102 starts the sound acquisition until aninstruction for presenting the combined contamination information isissued by the user.

The information presentation unit 108 includes, for example, a displaymember, such as a monitor having a touch panel function. The informationpresentation unit 108 converts the contamination information or thecombined contamination information as required and then presents theinformation, upon reception of, for example, an instruction from theuser. The information presentation unit 108 converts the informationabout the distance, direction, shape, and position of the contaminationincluded in the contamination information or the combined contaminationinformation into information that enables the user to recognize acontaminated area, such as a sound and a video. Examples of instructionsreceived by the information presentation unit 108 issued by the userinclude, for example, an instruction for starting the presentation ofthe contamination information or the combined contamination information,an instruction for ending the presentation thereof, an instruction forthe time period during which the contamination information is stored,and an instruction for the time period during which the contaminationinformation is deleted.

The system control unit 109 controls the entire information processingapparatus 101 via the communication bus 110. The system control unit 109implements each piece of processing (described below) by executing aprogram recorded in a storage unit, such as a nonvolatile memory (notillustrated).

The communication bus 110 connects the function units configuring theinformation processing apparatus 101 with each other.

The contaminated area detection and generation processing performed bythe information processing apparatus 101 will now be described withreference to FIG. 2 . FIG. 2 is a flowchart illustrating an example ofthe contaminated area detection and generation processing according tothe first exemplary embodiment. The information processing apparatus 101starts the contaminated area detection and generation processingillustrated in FIG. 2 when, for example, power of the apparatus isturned ON. The information processing apparatus 101 may also start thecontaminated area detection and generation processing illustrated inFIG. 2 upon reception of an instruction from the user.

In step S201, the system control unit 109 determines whether thepredetermined storage condition is specified via the informationpresentation unit 108. The predetermined storage condition to bespecified refers to, for example, the time period during which thecontamination information is stored. When the system control unit 109determines that the predetermined storage condition is specified (YES instep S201), the processing proceeds to step S202. In contrast, when thesystem control unit 109 determines that the predetermined storagecondition is not specified (NO in step S201), the processing proceeds tostep S204.

In step S202, the system control unit 109 sets the predetermined storagecondition. Upon completion of the storage condition setting, theprocessing proceeds to step S203.

In step S203, the system control unit 109 waits until the predeterminedstorage condition set in step S202 is satisfied. When the predeterminedstorage condition set in step S202 is satisfied, the processing proceedsto step S204.

In step S204, the system control unit 109 controls the sound acquisitionunit 102, the sound information generation unit 103, the sound directiondetection unit 104, and the facial state detection unit 106 to performdetection processing. The detection processing in step S204 will bedescribed in detail below with reference to FIG. 3 . Upon completion ofthe detection processing, the processing proceeds to step S205.

In step S205, referring to the result of the detection processingperformed in step S204, the system control unit 109 controls thecontamination information generation unit 107 to perform thecontamination information generation processing. The contaminationinformation generation processing in step S205 will be described indetail below with reference to FIG. 4 . Upon completion of thecontamination information generation processing, the processing proceedsto step S206.

In step S206, the system control unit 109 checks whether a condition forending the storage of the contamination information is set in thepredetermined storage condition and determines whether to end thestorage of the contamination information. The condition for ending thestorage of the contamination information refers to, for example, thefact that the time period during which the contamination information isstored has ended. When the system control unit 109 determines that thecondition for ending the storage of the contamination information issatisfied and determines to end the storage of the contaminationinformation (YES in step S206), the system control unit 109 completesthe contaminated area detection and generation processing. In contrast,when the system control unit 109 determines that the condition forending the storage of the contamination information is not satisfied anddetermines not to end the storage of the contamination information (NOin step S206), the processing proceeds to step S207.

In step S207, the system control unit 109 determines whether aninformation presentation instruction for requesting to present thecontamination information or the combined contamination information isissued from the user via the information presentation unit 108.

When the system control unit 109 determines that the informationpresentation instruction is issued (YES in step S207), the systemcontrol unit 109 completes the contaminated area detection andgeneration processing. In contrast, when the system control unit 109determines that no information presentation instruction is issued (NO instep S207), the processing returns to step S204.

The detection processing in step S204 in FIG. 2 will now be describedwith reference to FIG. 3 . FIG. 3 is a flowchart illustrating an exampleof the detection processing. The processing of the flowchart illustratedin FIG. 3 is implemented when the system control unit 109 controls eachfunction unit.

In step S301, the sound acquisition unit 102 determines whether a soundis acquired. When the sound acquisition unit 102 determines that nosound is acquired (NO in step S301), the sound acquisition unit 102repeats step S301 until a sound is acquired. When the sound acquisitionunit 102 determines that a sound is acquired (YES in step S301), theprocessing proceeds to step S302.

In step S302, the sound information generation unit 103 determineswhether a physiological sound of the body can be extracted from thesound acquired by the sound acquisition unit 102 in step S301. When thesound information generation unit 103 determines that a physiologicalsound of the body can be acquired (YES in step S302), the soundinformation generation unit 103 generates sound information in which aphysiological sound of the body is extracted. The processing thenproceeds to step S303. When the sound information generation unit 103determines that no physiology sound of the body can be acquired (NO instep S302), the processing returns to step S301.

In step S303, the sound information generation unit 103 generates soundvolume information indicating the sound volume of the physiologicalsound of the body extracted in step S302. Upon completion of the soundvolume information generation, the processing proceeds to step S304.

In step S304, the sound direction detection unit 104 detects thedirection of the physiological sound of the body extracted in step S302.Upon completion of detecting the sound direction, the processingproceeds to step S305.

In step S305, the imaging unit 105 captures an image of a subject. Uponcompletion of acquiring the captured image, the processing proceeds tostep S306.

In step S306, the facial state detection unit 106 determines whether aface exists in the direction of the sound detected by the sounddirection detection unit 104 in step S304 based on the image acquired instep S305. When the facial state detection unit 106 determines that aface exist in the direction of the detected sound (YES in step S306),the processing proceeds to step S307. When the facial state detectionunit 106 determines that no face exists in the direction of the detectedsound (NO in step S306), the processing proceeds to step S310.

In step S307, the facial state detection unit 106 detects the frontdirection of the face existing in the direction of the physiologicalsound of the body detected in step S304. Upon completion of thedetection of the front direction of the face, the processing proceeds tostep S308.

In step S308, the facial state detection unit 106 detects the distancebetween the face existing in the direction of the physiological sound ofthe body detected in step S304 and the imaging unit 105. Upon completionof the detection of the distance between the face and the imaging unit105, the processing proceeds to step S309.

In step S309, the facial state detection unit 106 detects the maskwearing state of the face existing in the direction of the physiologicalsound of the body detected in step S304. Upon completion of thedetection of the mask wearing state, the facial state detection unit 106completes the detection processing. Then, the processing proceeds tostep S205 in FIG. 2 .

The execution order of the processing in steps S307, S308, and S309 isnot limited to the example illustrated in FIG. 3 but may be in another(e.g., a random) order.

In step S310, the facial state detection unit 106 sets a facenon-detection flag. Upon completion of the face non-detection flagsetting, the facial state detection unit 106 completes the detectionprocessing. Then, the processing proceeds to step S205 in FIG. 2 .

The contamination information generation processing in step S205 in FIG.2 will now be described with reference to FIG. 4 . FIG. 4 is a flowchartillustrating an example of the contamination information generationprocessing. The processing of the flowchart illustrated in FIG. 4 isimplemented when the system control unit 109 controls each functionunit.

In step S401, the contamination information generation unit 107determines whether the face non-detection flag is set in step S310 inFIG. 3 . When the contamination information generation unit 107determines that the face non-detection flag is set (YES in step S401),the contamination information generation unit 107 completes thecontamination information generation processing without generating thecontamination information. Then, the processing proceeds to step S206 inFIG. 2 . In contrast, when the contamination information generation unit107 determines that the face non-detection flag is not set (NO in stepS401), the processing proceeds to step S402.

In step S402, the contamination information generation unit 107generates the contamination information in which a predetermined area inthe front direction of the face detected in step S307 in FIG. 3 is setas a contaminated area to which droplets adhere. Upon completion of thecontamination information generation, the processing proceeds to stepS403.

In step S403, the contamination information generation unit 107generates a position of the contaminated area based on the direction ofthe physiological sound of the body detected in step S304 in FIG. 3 andthe distance between the face and the imaging unit 105 detected in stepS308 in FIG. 3 . Upon completion of the inclusion of the generatedposition of the contaminated area in the contamination information, theprocessing proceeds to step S404.

In step S404, the contamination information generation unit 107determines whether the face wears a mask based on the mask wearing statedetected in step S309 in FIG. 3 . When the contamination informationgeneration unit 107 determines that the face wears a mask (YES in stepS404), the processing proceeds to step S405. In contrast, when thecontamination information generation unit 107 determines that the facedoes not wear a mask (NO in step S404), the processing proceeds to stepS406.

In step S405, the contamination information generation unit 107 changesthe contaminated area of the contamination information generated in stepS402. Examples of changes made in step S405 include decreasing thedistance of the contamination included in the contamination information,setting the direction of the contamination to the direction of theleakage from the gap between the mask and the face, and setting theshape of the contamination to the shape of the leakage from the gapbetween the mask and the face. Upon completion of the contaminationinformation change in step S405, the processing proceeds to step S406.

In step S406, the contamination information generation unit 107 changesthe contaminated area of the contamination information generated in stepS402 or the contaminated area of the contamination information changedin step S405 based on the sound volume information generated in stepS303 in FIG. 3 . Examples of changes made in step S406 includeincreasing the distance of the contamination information or enlargingthe shape of the contamination information with increasing sound volume.The changes made in step S406 may be performed based on the positions ofthe plurality of microphones included in the sound acquisition unit 102and the relative position of the contaminated area generated in stepS403. For example, with a large relative position and a large soundvolume, the distance of the contamination information or the shape ofthe contamination information may be increased. Upon completion of thecontamination information change in step S406, the processing proceedsto step S407.

In step S407, the contamination information generation unit 107 storesthe contamination information. Upon completion of the contaminationinformation storage, the contamination information generation unit 107completes the contamination information generation processing. Then, theprocessing proceeds to step S206 in FIG. 2 .

The contamination information deletion processing performed by theinformation processing apparatus 101 will now be described withreference to FIG. 5 . FIG. 5 is a flowchart illustrating an example ofthe contamination information deletion processing. The processing of theflowchart illustrated in FIG. 5 is implemented when the system controlunit 109 controls each function unit. The contamination informationdeletion processing in FIG. 5 is performed, for example, aftercompletion of the contamination information presentation processingdescribed below with reference to FIG. 6.

In step S501, the system control unit 109 determines whether apredetermined deletion condition is specified via the informationpresentation unit 108. The specified predetermined deletion conditionrefers to, for example, the time period during which the contaminationinformation is deleted. When the system control unit 109 determines thatthe predetermined deletion condition is specified (YES in step S501),the processing proceeds to step S502. When the system control unit 109determines that the predetermined deletion condition is not specified(NO in step S501), the processing proceeds to step S503.

In step S502, the system control unit 109 sets the specifiedpredetermined deletion condition. Upon completion of the deletioncondition setting in step S502, the processing proceeds to step S504.

In step S503, the system control unit 109 sets the time period since thesound acquisition is started until the combined contaminationinformation is presented, as the deletion condition. Upon completion ofthe deletion condition setting in step S503, the processing proceeds tostep S504.

In step S504, the contamination information generation unit 107 deletesthe contamination information satisfying the deletion condition set instep S502 or S503. Upon completion of the deletion of the contaminationinformation satisfying the deletion condition, the system control unit109 completes the contamination information deletion processing.

The contamination information presentation processing performed by theinformation processing apparatus 101 will now be described withreference to FIG. 6 . FIG. 6 is a flowchart illustrating an example ofthe contamination information presentation processing. The processing ofthe flowchart illustrated in FIG. 6 is implemented when the systemcontrol unit 109 controls each function unit. The contaminationinformation presentation processing in FIG. 6 is performed aftercompletion of the contaminated area detection and generation processingdescribed above with reference to FIG. 2 .

In step S601, the contamination information generation unit 107 combinespieces of the stored contamination information to generate the combinedcontamination information. Upon completion of the combined contaminationinformation generation, the processing proceeds to step S602.

In step S602, the information presentation unit 108 converts thecombined contamination information generated in step S601 and thenpresents the information. The information presentation unit 108 convertsthe information about the distance, direction, shape, and position ofthe contamination included in the combined contamination informationinto a user-recognizable image, and displays the image, therebypresenting the combined contamination information. Upon completion ofthe combined contamination information presentation, the processingproceeds to step S603.

In step S603, the information presentation unit 108 determines whetheran information presentation end instruction for requesting to end thepresentation of the combined contamination information is issued by theuser. When the information presentation unit 108 determines that noinformation presentation end instruction is issued (NO in step S603),the information presentation unit 108 continues the presentation of theinformation, and the processing proceeds to step S603. In contrast, whenthe information presentation unit 108 determines that the informationpresentation end instruction is issued (YES in step S603), theinformation processing apparatus 101 completes the contaminationinformation presentation processing.

FIG. 7 illustrates an example of the combined contamination informationpresented by the information processing apparatus 101. The exampleillustrated in FIG. 7 is the combined contamination informationgenerated by the information processing apparatus 101 installed on theceiling of a room 701. In the room 701, chairs 702, 703, 704, and 705are installed. As contaminated areas to which scattered droplets adhere,FIG. 7 illustrates areas 706, 707, 708, and 709, which correspond topieces of the stored contamination information. These areas arepresented as the combined contamination information as a result ofcombining pieces of the contamination information. The vertex of each ofthe areas 706 to 709 (center of a fan shape) is a face position, whichis indicating that the front directions of the faces of the personssitting on the chairs 702 to 705 are contaminated.

According to the first exemplary embodiment, the information processingapparatus 101 extracts a physiological sound of the body from the soundacquired by the sound acquisition unit 102, detects the state of theface existing in the direction of the physiological sound of the body,generates contaminated area information in which a predetermined areaaccording to the facial state is set as a contaminated area, andpresents the information. By detecting and presenting a contaminatedarea based on the direction of the physiological sound of the body, suchas the sound of an utterance, sneeze, cough, blow, and exhalation, andthe facial state in this way, it becomes possible to recognize acontaminated area to which droplets scattered from a person adhere. Itis also possible to recognize a contaminated area to which dropletsscattered from a person adhere, enabling efficient cleaning.

The information presentation unit 108 including, for example, operationbuttons and a projector may receive an operation instruction from theuser, convert the contamination information or the combinedcontamination information into an image, and project the image forpresentation. The conversion of the contamination information or thecombined contamination information performed by the informationpresentation unit 108 may include the conversion into mappinginformation in which a video is mapped on an object.

When the information presentation unit 108 maps a video on the object,the shape of the object may be extracted from, for example, the imagecaptured by the imaging unit 105 or may be preregistered. In this case,in step S602 (contamination information presentation processing) in FIG.6 , the information presentation unit 108 needs to convert the video ofthe combined contamination information generated in step S601 into themapping information in which a video is mapped on an object, and projectthe information for presentation.

By converting the contaminated area into a video and projecting thevideo to an object for presentation in this way, it becomes possible tovisually recognize a contaminated area to which droplets scattered froma person adhere. It is also possible to visually recognize acontaminated area to which droplets scattered from a person adhere,enabling efficient cleaning.

FIG. 8 is a block diagram illustrating an example configuration of aninformation processing system 801 according to a second exemplaryembodiment. Referring to FIG. 8 , components having identical functionsto those illustrated in FIG. 1 are assigned the same reference numeralsand duplicated descriptions thereof will be omitted. An informationprocessing system 801 includes an information processing apparatus 802and an information presentation apparatus 803. The informationprocessing apparatus 802 includes a sound acquisition unit 102, a soundinformation generation unit 103, a sound direction detection unit 104,an imaging unit 105, a facial state detection unit 106, a contaminationinformation generation unit 107, a system control unit 109, and acommunication bus 110. An information presentation apparatus 803 is, forexample, augmented reality (AR) glasses. The information presentationapparatus 803 includes an information presentation unit 804.

The information presentation unit 804 includes, for example, operationbuttons, a loud speaker, a transmissive monitor, a gyroscope sensor, andan acceleration sensor. Upon reception of an instruction from anoperating user, for example, the information presentation unit 804suitably converts the contamination information or the combinedcontamination information and displays the resultant information forpresentation. The conversion of the contamination information or thecombined contamination information performed by the informationpresentation unit 804 refers to, for example, the conversion into asound and a video. The conversion of the contamination information orthe combined contamination information performed by the informationpresentation unit 804 includes the conversion into mapping informationin which a video is mapped on an object. The shape of the object whenthe information presentation unit 804 maps a video on an object may be,for example, extracted from the image captured by the imaging unit 105or preregistered.

The information presentation unit 804 having a self-position estimationfunction can also generate self-position estimation information in whichthe position of the information presentation apparatus 803 is estimated.The information presentation apparatus 803 performs the self-positionestimation, for example, by detecting the information presentationapparatus 803 from the image captured by the imaging unit 105 andcalculating the self-position based on information from the gyroscopesensor and the acceleration sensor. Examples of instructions from theuser received by the information presentation unit 804 include aninstruction for starting the presentation of the contaminationinformation or the combined contamination information, an instructionfor ending the presentation thereof, an instruction for the time periodduring which the contamination information is stored, and an instructionfor the time period during which the contamination information isdeleted.

The contaminated area detection and generation processing performed bythe information processing system 801 according to the second exemplaryembodiment is similar to the contaminated area detection and generationprocessing according to the first exemplary embodiment described abovewith reference to FIGS. 2 to 4 , and therefore descriptions thereof willbe omitted. Also, the contamination information deletion processingperformed by the information processing system 801 according to thesecond exemplary embodiment is similar to the contamination informationdeletion processing according to the first exemplary embodimentdescribed above with reference to FIG. 5 , and descriptions thereof willbe omitted.

The contamination information presentation processing performed by theinformation processing system 801 will now be described with referenceto FIG. 9 .

FIG. 9 is a flowchart illustrating an example of the contaminationinformation presentation processing. The processing of the flowchartillustrated in FIG. 9 is implemented when the system control unit 109controls each function unit. The contamination information presentationprocessing in FIG. 9 is performed after completion of the contaminatedarea detection and generation processing described above with referenceto FIG. 2 .

In step S901, the information presentation unit 804 estimates theposition of the information presentation apparatus 803 and generates theself-position estimation information.

In step S902, based on the self-position estimation informationgenerated by the information presentation unit 804 in step S901, theinformation presentation unit 804 converts the information about theposition of the contaminated area out of the contamination informationstored by the contamination information generation unit 107 into arelative position that is relative to the self-position estimationinformation. Upon completion of the conversion of the position of thecontaminated area in the contamination information into a relativeposition, the processing proceeds to step S903.

In step S903, the contamination information generation unit 107 combinespieces of the contamination information as a result of the conversion instep S902 to generate the combined contamination information. Uponcompletion of the combined contamination information generation, theprocessing proceeds to step S904.

In step S904, the information presentation unit 804 displays thecombined contamination information generated in step S903 forpresentation. The information presentation unit 804 may also output,from a speaker, a warning sound with a larger sound volume for a closerrelative position for presentation, based on the relative position tothe position of the contaminated area as a result of the conversion instep S902.

In step S905, the information presentation unit 804 determines whetheran information presentation end instruction for requesting to end thepresentation of the combined contamination information is issued by theuser. When the information presentation unit 804 determines that noinformation presentation end instruction is issued (NO in step S905),the processing returns to step S901. When the information presentationunit 804 determines that the information presentation end instruction isissued (YES in step S905), the system control unit 109 completes thecontamination information presentation processing. Upon completion ofthe contamination information presentation processing described abovewith reference to FIG. 9 , the information processing system 801performs the contamination information deletion processing describedabove with reference to FIG. 5 .

According to the second exemplary embodiment, the information processingapparatus 802 extracts a physiological sound of the body from the soundacquired by the sound acquisition unit 102, detects the state of theface existing in the direction of the physiological sound of the body,and generates contaminated area information in which a predeterminedarea according to the facial state is set as a contaminated area. Then,based on the self-position estimation information generated by theinformation presentation unit 804, the information processing apparatus802 presents the contaminated area in a form that makes it easier toperform estimation from the self-position of the user who wears theinformation presentation apparatus 803. This enables recognizing indetail the contaminated area to which droplets scattered from a personadhere, making it possible to efficiently clean the contaminated area.

In a third exemplary embodiment, the stored contamination informationaccording to the movement of the object is updated when an objectexisting in a certain contaminated area is moved. FIG. 10 is a blockdiagram illustrating an example configuration of an informationprocessing apparatus 1001 according to the third exemplary embodiment.Referring to FIG. 10 , components having identical functions to thoseillustrated in FIG. 1 are assigned the same reference numerals andduplicated descriptions thereof will be omitted. The informationprocessing apparatus 1001 includes a sound acquisition unit 102, a soundinformation generation unit 103, a sound direction detection unit 104,an imaging unit 105, a facial state detection unit 106, an informationpresentation unit 108, a system control unit 109, a communication bus110, an object movement detection unit 1002, and a contaminationinformation generation unit 1003.

The object movement detection unit 1002 detects a moving object in animage captured by the imaging unit 105. The object movement detectionunit 1002 detects the range where the detected moving object has existedbefore the movement and the range where the detected moving objectexists after the movement, as movement information for the movingobject. A known movement detection technique is used for the movingobject detection by the object movement detection unit 1002. Thetechnique analyzes a captured image and then performs pattern matchingon an object and tracks the object by using a motion vector.

The contamination information generation unit 1003 generatescontamination information in which a predetermined area according to thefacial state is set as a contaminated area to which scattered dropletsadhere based on the facial state detected by the facial state detectionunit 106. The contamination information generation unit 1003 alsoupdates the contents of the generated contamination information based onthe movement information detected by the object movement detection unit1002. The update of the movement information detected by the objectmovement detection unit 1002 and the contamination information generatedby the contamination information generation unit 1003 will now bedescribed in detail.

The contaminated area detection and generation processing performed bythe information processing apparatus 1001 will now be described withreference to FIG. 11 . FIG. 11 is a flowchart illustrating an example ofthe contaminated area detection and generation processing according tothe third exemplary embodiment. The information processing apparatus1001 starts the contaminated area detection and generation processingillustrated in FIG. 11 , for example, when power of the apparatus isturned ON. The information processing apparatus 1001 may start thecontaminated area detection and generation processing illustrated inFIG. 11 upon reception of an instruction from the user.

In step S1101, the system control unit 109 determines whether thepredetermined storage condition is specified via the informationpresentation unit 108. The predetermined storage condition to bespecified refers to, for example, the time period during which thecontamination information is stored. When the system control unit 109determines that the predetermined storage condition is specified (YES instep S1101), the processing proceeds to step S1102. In contrast, whenthe system control unit 109 determines that the predetermined storagecondition is not specified (NO in step S1101), the processing proceedsto step S1104.

In step S1102, the system control unit 109 sets the predeterminedstorage condition. Upon completion of the storage condition setting, theprocessing proceeds to step S1103.

In step S1103, the system control unit 109 waits until the predeterminedstorage condition set in step S1102 is satisfied. When the predeterminedstorage condition is satisfied, the processing proceeds to step S1104.

In step S1104, the system control unit 109 controls the soundacquisition unit 102, the sound information generation unit 103, thesound direction detection unit 104, and the facial state detection unit106 to perform the detection processing. The detection processing instep S1104 is similar in detail to the detection processing according tothe first exemplary embodiment described above with reference to FIG. 3, and descriptions thereof will be omitted. Upon completion of thedetection processing, the processing proceeds to step S1105.

In step S1105, referring to the result of the detection processingperformed in step S1104, the system control unit 109 controls thecontamination information generation unit 1003 to perform thecontamination information generation processing. The contaminationinformation generation processing in step S1105 is similar in detail tothe contamination information generation processing according to thefirst exemplary embodiment described above with reference to FIG. 4 ,and thus descriptions thereof will be omitted. Upon completion of thecontamination information generation processing, the processing proceedsto step S1106.

In step S1106, the system control unit 109 controls the object movementdetection unit 1002 to perform the moving object detection processingand determines whether a moving object is detected in the image capturedby the imaging unit 105. In the moving object detection processing, theobject movement detection unit 1002 detects a moving object in the imagecaptured by the imaging unit 105 and detects the area where the movingobject has existed before the movement and the area where the movingobject exists after the movement, as movement information for the movingobject. The movement information for the detected moving object istransmitted to the contamination information generation unit 1003. Asdescribed above, a known movement detection technique may be used forthe moving object detection by the object movement detection unit 1002.

When the system control unit 109 determines that a moving object isdetected in the captured image (YES in step S1106), the processingproceeds to step S1107. In contrast, when the system control unit 109determines that no moving object is detected in the captured image (NOin step S1106), the processing proceeds to step S1108.

The movement information detected by the object movement detection unit1002 will now be described with reference to FIGS. 12A to 12C. FIG. 12Aillustrates a state where a chair 1201 exists at a predeterminedposition in the room 701. FIG. 12B illustrates a state where the chair1201 is moved to another position. FIG. 12C illustrates an area (firstexistence area) 1211 where the chair 1201 has existed before themovement, and an area (second existence area) 1212 where the chair 1212exists after the movement. When the chair 1201 is moved in this way, theobject movement detection unit 1002 detects the first existence area1211 and the second existence area 1212 illustrated in FIG. 12C asmovement information for the chair 1201 (moving object) and thentransmits the information to the contamination information generationunit 1003.

Referring back to FIG. 11 , in step S1107, the system control unit 109controls the contamination information generation unit 1003 to updatethe contamination information. In updating the contaminationinformation, the contamination information generation unit 1003 updatesthe contamination information based on the movement information detectedin step S1108. The update of the contamination information generated bythe contamination information generation unit 1003 will be describedwith reference to FIGS. 13A to 13E.

FIG. 13A illustrates a state where contamination information similar tothat in the example illustrated in FIG. 7 is generated in a state wherethe chair 1201 further exists in the room 701 described above withreference to FIG. 7 . FIG. 13A illustrates a state where thecontaminated area 708 partly overlaps with the chair 1201.

FIG. 13B illustrates a state where the chair 1201 has been moved fromthe state where the contamination information is generated asillustrated in FIG. 13A to another position. As illustrated in FIG. 13B,the chair 1201, having existed in a certain contaminated area before themovement, exists at a position out of the contaminated area after themovement. In this state, no contamination information is generated inthe area (second existence area) where the contaminated chair 1201exists after the movement. Thus, no correct contaminated area can bepresented to the user.

Accordingly, when at least a part of the first existence area of thedetected moving object overlaps with the contaminated area, theinformation processing apparatus 1001 generates new contaminationinformation in which the second existence area of the moving object isset as a contaminated area. In this example, as illustrated in FIG. 13C,the information processing apparatus 1001 generates new contaminationinformation in which the destination existence area (second existencearea) 1301 of the moving object that has existed in a certaincontaminated area is set as a contaminated area. This enables presentingto the user that, even when the moving object that has existed in acertain contaminated area is moved, the movement destination is acontaminated area.

In a state illustrated in FIG. 13B, the area of the contaminated area708 where the chair 1201 has existed before the movement still remainsto be a contaminated area. However, since the chair 1201 has been moved,it is thought that the area (first existence area) where the chair 1201has existed before the movement is no longer a contaminated area.

Accordingly, when the object that has existed in a certain contaminatedarea is moved, the information processing apparatus 1001 may update thecontamination information so that the portion of the first existencearea overlapping with the moving object is excluded from thecontaminated areas. For example, as illustrated in FIG. 13D, theinformation processing apparatus 1001 updates the contaminationinformation to change the contaminated area 708 to a contaminated area1311 so that the existence area (first existence area) of the movingobject (that has existed in a certain contaminated area) before themovement is excluded from the contaminated areas. This enablespresenting to the user that, when the moving object that has existed ina certain contaminated area is moved, the existence area of the objectbefore the movement is no longer a contaminated area. Although, in FIG.13D, the shape of the contaminated area is changed so that the firstexistence area of the moving object (that has existed in a certaincontaminated area) is excluded from the contaminated areas, someembodiments are not limited thereto. For example, the informationprocessing apparatus 1001 can also update the contamination informationto new contamination information by adding information indicating thatthe first existence area of the moving object (that has existed in acertain contaminated area) is no longer a contaminated area, to thecontamination information, while maintaining the shape of thecontaminated area.

When an object that has existed in a certain contaminated area is moved,the information processing apparatus 1001 may generate new contaminationinformation in which the locus range of the movement of the object isset as a contaminated area.

In this example case, as illustrated in FIG. 13E, the informationprocessing apparatus 1001 generates new contamination information inwhich a locus range 1323 of the chair 1201 having been moved from afirst existence area 1321 to a second existence area 1322 is set as acontaminated area. When an object that has existed in a certaincontaminated area is moved, this processing enables presenting a movingpath that may possibly be newly contaminated to the user, as acontaminated area.

Upon completion of the contamination information update based on themovement information in step S1107, as described above, the processingproceeds to step S1108.

In step S1108, the system control unit 109 confirms whether a conditionfor ending the storage of the contamination information is set as apredetermined storage condition and determines whether to end thestorage of the contamination information.

When the system control unit 109 determines that the condition forending the storage of the contamination information is satisfied anddetermines to end the storage of the contamination information (YES instep S1108), the system control unit 109 completes the contaminated areadetection and generation processing. In contrast, when the systemcontrol unit 109 determines that the condition for ending the storage ofthe contamination information is not satisfied and determines not to endthe storage of the contamination information (NO in step S1108), theprocessing proceeds to step S1109.

In step S1109, the system control unit 109 determines whether aninformation presentation instruction for requesting to present thecontamination information or the combined contamination information isissued from the user via the information presentation unit 108. When thesystem control unit 109 determines that the information presentationinstruction is issued (YES in step S1109), the system control unit 109completes the contaminated area detection and generation processing. Incontrast, when the system control unit 109 determines that noinformation presentation instruction is issued (NO in step S1109), theprocessing returns to step S1104.

According to the third exemplary embodiment, the information processingapparatus 1001 updates the contamination information according to themovement information for an object, thereby enabling the user toreliably recognize contaminated areas even after the movement of anobject that has existed in a certain contaminated area.

The third exemplary embodiment has been described above centering on anexample operation where, when an object existing in a certaincontaminated area is moved, the stored contamination information isupdated according to the movement of the object.

A fourth exemplary embodiment will be described below centering on anexample operation of storing video data during the movement and imagedata before and after the movement when the object existing in thecontaminated area is moved. The fourth exemplary embodiment is similarto the above-described third exemplary embodiment except for thecontaminated area detection and generation processing performed by theinformation processing apparatus 1001, and descriptions thereof will beomitted. The contaminated area detection and generation processingaccording to the fourth exemplary embodiment will now be described.

FIG. 14 is a flowchart illustrating an example of the contaminated areadetection and generation processing according to the fourth exemplaryembodiment. The information processing apparatus 1001 starts thecontaminated area detection and generation processing illustrated inFIG. 14 , for example, when power of the apparatus is turned ON. Theinformation processing apparatus 1001 may start the contaminated areadetection and generation processing illustrated in FIG. 14 uponreception of an instruction from the user.

In step S1401, the system control unit 109 determines whether thepredetermined storage condition is specified via the informationpresentation unit 108. The predetermined storage condition to bespecified refers to, for example, the time period during which thecontamination information is stored. When the system control unit 109determines that the predetermined storage condition is specified (YES instep S1401), the processing proceeds to step S1402. In contrast, whenthe system control unit 109 determines that the predetermined storagecondition is not specified (NO in step S1401), the processing proceedsto step S1404.

In step S1402, the system control unit 109 sets the predeterminedstorage condition. Upon completion of the storage condition setting, theprocessing proceeds to step S1403.

In step S1403, the system control unit 109 waits until the predeterminedstorage condition set in step S1402 is satisfied. When the predeterminedstorage condition is satisfied, the processing proceeds to step S1404.

In step S1404, the system control unit 109 controls the soundacquisition unit 102, the sound information generation unit 103, thesound direction detection unit 104, and the facial state detection unit106 to perform detection processing. The detection processing in stepS1404 is similar in detail to the detection processing according to thefirst exemplary embodiment described above with reference to FIG. 3 ,and descriptions thereof will be omitted. Upon completion of thedetection processing, the processing proceeds to step S1405.

In step S1405, referring to the result of the detection processingperformed in step S1404, the system control unit 109 controls thecontamination information generation unit 1003 to perform thecontamination information generation processing. The contaminationinformation generation processing in step S1405 is similar in detail tothe contamination information generation processing according to thefirst exemplary embodiment described above with reference to FIG. 4 ,and descriptions thereof will be omitted. Upon completion of thecontamination information generation processing, the processing proceedsto step S1406.

In step S1406, the system control unit 109 controls the object movementdetection unit 1002 to perform the moving object detection processingand determines whether a moving object is detected in the image capturedby the imaging unit 105. When the system control unit 109 determinesthat a moving object is detected in the captured image (YES in stepS1406), the processing proceeds to step S1407. In contrast, when thesystem control unit 109 determines that no moving object is detected inthe captured image (NO in step S1406), the processing proceeds to stepS1408.

In step S1407, since a moving object was detected in step S1406, thesystem control unit 109 stores the video data in which the objectcaptured by the imaging unit 105 is moving and the image data before andafter the movement of the object. Thus, when a moving object isdetected, the system control unit 109 stores not only the updatedcontamination information according to the third exemplary embodimentbut also the scene of the movement, as video or image data. When thecontaminated area is cleaned, this processing enables presenting videoor image information as supplementary information together with thecontaminated area to the user. Regarding the storage of the video dataand the image data, either one or both of these pieces of data may bestored according to a storage mode preselected by the user. Uponcompletion of the storage of the video data of the moving object and/orthe image data before and after the movement thereof in step S1407, theprocessing proceeds to step S1408.

In step S1408, the system control unit 109 confirms whether a conditionfor ending the storage of the contamination information is set as apredetermined storage condition, and determines whether to end thestorage of the contamination information.

When the system control unit 109 determines that the condition forending the storage of the contamination information is satisfied anddetermines to end the storage of the contamination information (YES instep S1408), the system control unit 109 completes the contaminated areadetection and generation processing. In contrast, when the systemcontrol unit 109 determines that the condition for ending the storage ofthe contamination information is not satisfied and determines not to endthe storage of the contamination information (NO in step S1408), theprocessing proceeds to step S1409.

In step S1409, the system control unit 109 determines whether aninformation presentation instruction for requesting to present thecontamination information or the combined contamination information isissued from the user via the information presentation unit 108. When thesystem control unit 109 determines that the information presentationinstruction is issued (YES in step S1409), the system control unit 109completes the contaminated area detection and generation processing. Incontrast, when the system control unit 109 determines that noinformation presentation instruction is issued (NO in step S1409), theprocessing returns to step S1404.

According to the fourth exemplary embodiment, the information processingapparatus 1001 stores the video data and/or the image data according tothe movement of an object, thereby enabling the user to reliablyrecognize contaminated areas even after the movement of an object thathas existed in a certain contaminated area.

FIG. 15 is a block diagram illustrating an example configuration of acleaning apparatus 1501 according to a fifth exemplary embodiment.Referring to FIG. 15 , components having identical functions to thoseillustrated in FIG. 1 are assigned the same reference numerals andduplicated descriptions thereof will be omitted. The cleaning apparatus1501 includes a sound acquisition unit 102, a sound informationgeneration unit 103, a sound direction detection unit 104, an imagingunit 105, a facial state detection unit 106, a contamination informationgeneration unit 107, an information presentation unit 108, acommunication bus 110, a system control unit 1502, and a cleaningoperation unit 1504. The term “cleaning” comprehensively means overallactions intending to suppress bacteria or viruses (e.g., bacteriaelimination, antisepsis, sterilization, antibacterial actions, virusremoval, virus killing, and disinfection).

The system control unit 1502 having a similar function to the systemcontrol unit 109 according to the first exemplary embodiment controlsthe entire cleaning apparatus 1501 via the communication bus 110.

The system control unit 1502 implements each piece of processing(described below) by executing a program recorded in a storage unit suchas a nonvolatile memory (not illustrated). The system control unit 1502includes a cleaning operation control unit 1503.

The cleaning operation control unit 1503 controls operation of thecleaning operation unit 1504 via the communication bus 110 by executinga program recorded in a storage unit such as a nonvolatile memory (notillustrated).

The cleaning operation unit 1504 is a cleaning machine controlled by thecleaning operation control unit 1503. The cleaning operation unit 1504is, for example, a nebulizer for spraying the cleaning agent such asalcohol in a mist form, or an irradiator for radiating light having acleaning effect, such as an ultraviolet ray. The cleaning operation unit1504 may also be a self-propelled robot cleaner. According to the fifthexemplary embodiment, the cleaning operation unit 1504 is, for example,a cleaning agent nebulizer installed on the ceiling of the room 701,capable of controlling the spray angle and the spray time duration. Thespray angle of the cleaning agent and the spray range thereof in theroom 701 are associated with each other as data in a storage area (notillustrated). The spray time during which the contaminated area can besufficiently cleaned is to be acquired in a prior experiment and isstored in a storage area (not illustrated) as data.

Operation of the cleaning apparatus 1501 will now be described.

The cleaning apparatus 1501 performs the contaminated area detection andcontamination information generation processing. The contaminated areadetection and generation processing performed by the cleaning apparatus1501 is similar to the contaminated area detection and generationprocessing according to the first exemplary embodiment described abovewith reference to FIGS. 2 to 4 , and thus descriptions thereof will beomitted.

The cleaning operation performed by the cleaning apparatus 1501 based onthe contamination information generated in the contaminated areadetection and generation processing will be described with reference toFIGS. 16 and 17 . FIG. 16 is a flowchart illustrating an example of thecleaning operation processing. For example, when a predetermined timeduration has elapsed since the cleaning apparatus 1501 completes thecontaminated area detection and generation processing, the cleaningapparatus 1501 automatically starts the cleaning operation processingillustrated in FIG. 16 . The cleaning apparatus 1501 may also start thecleaning operation processing illustrated in FIG. 16 upon reception ofan instruction from the user.

When the cleaning apparatus 1501 starts the cleaning operationprocessing, then in step S1601, the cleaning operation control unit 1503determines the operation contents of the cleaning operation unit 1504based on the contamination information generated by the contaminationinformation generation unit 107. Examples of operation contents include,for example, the spray angle and the spray time of the cleaning agentused in the cleaning operation unit 1504. In this case, the cleaningoperation control unit 1503 determines the spray angle such that thespray range of the cleaning agent sufficiently covers the shape of thecontaminated area included in the contamination information. Thecleaning operation control unit 1503 also determines the spray time ofthe cleaning agent so that the contaminated area is sufficientlycleaned. Upon completion of the determination of the operation contentsof the cleaning operation unit 1504, the processing proceeds to stepS1602.

In step S1602, the cleaning operation control unit 1503 controls thecleaning operation unit 1504 to perform cleaning based on the operationcontents determined in step S1601. The cleaning operation control unit1503 instructs the cleaning operation unit 1504 to adjust the sprayangle of the cleaning operation unit 1504 and to spray the cleaningagent for a predetermined time duration based on the determinedoperation contents. Upon completion of the control of the cleaningoperation unit 1504 according to the operation contents, the processingproceeds to step S1603.

In step S1603, the cleaning operation control unit 1503 determineswhether the cleaning operation unit 1504 has completed all of theoperation contents determined in step S1601. When the cleaning operationcontrol unit 1503 determines that any of the operation contents isincomplete (NO in step S1603), the processing returns to step S1602.Then, the cleaning operation control unit 1503 continues the control ofthe operation of the cleaning operation unit 1504. In contrast, when thecleaning operation control unit 1503 determines that all of theoperation contents determined in step S1601 are completed (YES in stepS1603), the cleaning operation control unit 1503 completes the cleaningoperation processing.

FIG. 17 illustrates a cleaning operation performed by the cleaningapparatus 1501 illustrated in FIG. 16 with reference to an example wherethe cleaning apparatus 1501 is installed in a room 1701. The soundacquisition unit 102 and the cleaning operation unit 1504 included inthe cleaning apparatus 1501 are visible from the outside, and areconnected with other components of the cleaning apparatus 1501 via acommunication unit (not illustrated). A contaminated area 1702 indicatesthe shape of the contaminated area included in the contaminationinformation generated in the contaminated area detection and generationprocessing and then stored. A spray range 1703 indicates the shape ofthe spray range of the cleaning agent obtained by the operation of thecleaning operation unit 1504 described in steps S1601 and S1602 in FIG.16 .

In step S1601 in FIG. 16 , the cleaning operation control unit 1503determines the spray angle of the cleaning operation unit 1504 so thatthe spray range 1703 of the cleaning agent sufficiently covers theentire contaminated area 1702. In step S1602 in FIG. 16 , the cleaningoperation control unit 1503 controls the spray angle of the cleaningoperation unit 1504 to spray the cleaning agent based on the spray angledetermined in step S1601. This enables the cleaning apparatus 1501 tospray the cleaning agent to the entire contaminated area 1702. This alsoenables restricting the spraying of the cleaning agent fornon-contaminated areas outside the contaminated area 1702. By performingsuch processing, the cleaning apparatus 1501 can automatically clean theareas contaminated by droplets while restricting the amount of thecleaning agent.

Prior to the cleaning operation processing according to the presentexemplary embodiment, the cleaning apparatus 1501 may or may not performthe contamination information presentation processing described abovewith reference to FIG. 6 similar to that in the first exemplaryembodiment. When the contamination information presentation processinghas been performed, the cleaning apparatus 1501 starts the cleaningoperation processing illustrated in FIG. 16 in a state where thecontaminated area 1702 illustrated in FIG. 17 is presented. The cleaningapparatus 1501 completes the contamination information presentationprocessing after completion of the cleaning operation processingillustrated in FIG. 16 . Such processing makes it easier for the user todetermine whether the contaminated area has been cleaned by the cleaningapparatus 1501.

According to the fifth exemplary embodiment, the cleaning apparatus 1501detects a contaminated area and then automatically cleans only the area,making it possible to recognize the contaminated area to which dropletsscattered from a person adhere and efficiently performing the cleaning.

The cleaning apparatus according to a sixth exemplary embodimentcontrols a plurality of cleaning operation units to perform the cleaningon a plurality of contaminated areas. FIG. 18 is a block diagramillustrating an example configuration of a cleaning apparatus 1801according to the sixth exemplary embodiment. Referring to FIG. 18 ,components having identical functions to those illustrated in FIG. 1 areassigned the same reference numerals and duplicated descriptions thereofwill be omitted. The cleaning apparatus 1801 includes a soundacquisition unit 102, a sound information generation unit 103, a sounddirection detection unit 104, a facial state detection unit 106, acontamination information generation unit 107, an informationpresentation unit 108, a communication bus 110, an imaging unit 1802,and a system control unit 1803. The cleaning apparatus 1801 alsoincludes a first cleaning operation unit 1805, a second cleaningoperation unit 1806, a communication unit 1807, a network connectionunit 1808, a data acquisition unit 1809, a priority determination unit1810, and a cleaning result determination unit 1811.

The imaging unit 1802 having a similar function to the imaging unit 105according to the first exemplary embodiment captures a subject togenerate an image of the subject. The imaging unit 1802 can acquire bodytemperature information for the subject.

The system control unit 1803 having a similar function to the systemcontrol unit 109 according to the first exemplary embodiment controlsthe entire cleaning apparatus 1801 via the communication bus 110.

The system control unit 1803 implements each piece of processing(described below) by executing a program recorded in a storage unit suchas a nonvolatile memory (not illustrated). The system control unit 1803also includes a cleaning operation control unit 1804.

The cleaning operation control unit 1804 controls the operations of thefirst cleaning operation unit 1805 and the second cleaning operationunit 1806 via the communication bus 110 by executing a program recordedin a storage unit such as a nonvolatile memory (not illustrated).

The first cleaning operation unit 1805 and the second cleaning operationunit 1806 are cleaning machines controlled by the cleaning operationcontrol unit 1804. For example, each of the first cleaning operationunit 1805 and the second cleaning operation unit 1806 is, for example, anebulizer for spraying a cleaning agent, such as alcohol, in a mistform, or an irradiator for radiating light having a cleaning effect,such as an ultraviolet ray. Each of the first cleaning operation unit1805 and the second cleaning operation unit 1806 may also be aself-propelled robot cleaner. According to the sixth exemplaryembodiment, for example, the first cleaning operation unit 1805 is acleaning agent nebulizer installed on the ceiling of the room 1701,capable of controlling the spray angle and the spray time duration. Thesecond cleaning operation unit 1806 is a self-propelled robot capable ofirradiating the floor surface with an ultraviolet ray.

The communication unit 1807 is a communication module capable ofconnecting with a network via the network connection unit 1808.

The network connection unit 1808 is an interface for connecting thecommunication unit 1807 to the network.

The data acquisition unit 1809 connects with a network via thecommunication unit 1807 and the network connection unit 1808 to acquiredata for a subject who entered the room 1701. The data acquisition unit1809 stores the acquired data for the subject and the contaminationinformation generated by the contamination information generation unit107 in an associated way. Examples of the data for the subject include,for example, medical conditions of the subject (or medical history,symptoms, or doctor's diagnosis result). The data acquisition unit 1809also detects a subject from the image captured by the imaging unit 1802and stores the body temperature information for the subject and thecontamination information in an associated way.

The priority determination unit 1810 determines the cleaning priorityfor each contaminated area based on the data stored in the dataacquisition unit 1809. The cleaning result determination unit 1811determines a cleaning result indicating whether cleaning operation issufficient based on the priority determined by the prioritydetermination unit 1810 and the operation contents for cleaning by thefirst cleaning operation unit 1805 and the second cleaning operationunit 1806.

The contaminated area detection and generation processing performed bythe cleaning apparatus 1801 will now be described with reference to FIG.19 . FIG. 19 is a flowchart illustrating an example of the contaminatedarea detection and generation processing according to the sixthexemplary embodiment. The cleaning apparatus 1801 starts thecontaminated area detection and generation processing illustrated inFIG. 19 , for example, when power of the apparatus is turned ON. Thecleaning apparatus 1801 may start the contaminated area detection andgeneration processing illustrated in FIG. 19 upon reception of aninstruction from the user.

In step S1901, the system control unit 1803 determines whether apredetermined storage condition is specified via the informationpresentation unit 108. The predetermined storage condition to bespecified refers to, for example, the time period during which thecontamination information is stored. When the system control unit 1803determines that the predetermined storage condition is specified (YES instep S1901), the processing proceeds to step S1902. In contrast, whenthe system control unit 1803 determines that the predetermined storagecondition is not specified (NO in step S1901), the processing proceedsto step S1904.

In step S1902, the system control unit 1803 sets the predeterminedstorage condition. Upon completion of the storage condition setting, theprocessing proceeds to step S1903.

In step S1903, the system control unit 1803 waits until thepredetermined storage condition set in step S1902 is satisfied. When thepredetermined storage condition is satisfied, the processing proceeds tostep S1904.

In step S1904, the system control unit 1803 performs detectionprocessing by controlling the sound acquisition unit 102, the soundinformation generation unit 103, the sound direction detection unit 104,and the facial state detection unit 106. The detection processing instep S1904 is similar in detail to the detection processing according tothe first exemplary embodiment described above with reference to FIG. 3, and descriptions thereof will be omitted. Upon completion of thedetection processing, the processing proceeds to step S1905.

In step S1905, referring to the result of the detection processingperformed in step S1104, the system control unit 1803 controls thecontamination information generation unit 107 to perform thecontamination information generation processing. The contaminationinformation generation processing in step S1905 is similar in detail tothe contamination information generation processing according to thefirst exemplary embodiment described above with reference to FIG. 4 ,and descriptions thereof will be omitted. Upon completion of thecontamination information generation processing, the processing proceedsto step S1906.

In step S1906, the system control unit 1803 controls the dataacquisition unit 1809 to perform data acquisition processing. The dataacquisition processing in step S1906 will be described in detail belowwith reference to FIG. 20 . Upon completion of the data acquisitionprocessing, the processing proceeds to step S1907.

In step S1907, the system control unit 1803 confirms whether a conditionfor ending the storage of the contamination information is set as apredetermined storage condition and determines whether to end thestorage of the contamination information. When the system control unit1803 determines that the condition for ending the storage of thecontamination information is satisfied and determines to end the storageof the contamination information (YES in step S1907), the system controlunit 1803 completes the contaminated area detection and generationprocessing. In contrast, when the system control unit 1803 determinesthat the condition for ending the storage of the contaminationinformation is not satisfied and determines not to end the storage ofthe contamination information (NO in step S1907), the processingproceeds to step S1908.

In step S1908, the system control unit 1803 determines whether aninformation presentation instruction for requesting to present thecontamination information or the combined contamination information isissued from the user via the information presentation unit 108. When thesystem control unit 1803 determines that the information presentationinstruction is issued (YES in step S1908), the system control unit 1803completes the contaminated area detection and generation processing. Incontrast, when the system control unit 1803 determines that noinformation presentation instruction is issued (NO in step S1908), theprocessing returns to step S1904.

The data acquisition processing in step S1906 in FIG. 19 will now bedescribed with reference to FIG. 20 . FIG. 20 is a flowchartillustrating an example of the data acquisition processing. The dataacquisition unit 1809 performs the processing of the flowchartillustrated in FIG. 20 under the control of the system control unit1803.

When the data acquisition processing is started, then in step S2001, thedata acquisition unit 1809 detects a subject. This subject is identicalto the subject having been subjected to the detection processing insteps S1904 and S1905. Upon completion of the detection of a subject,the processing proceeds to step S2002.

In step S2002, the data acquisition unit 1809 acquires the bodytemperature information for the subject from the imaging unit 1802. Uponcompletion of the acquisition of the body temperature information forthe subject, the processing proceeds to step S2003.

In step S2003, the data acquisition unit 1809 connects with the networkvia the communication unit 1807 and the network connection unit 1808.Upon completion of network connection, the processing proceeds to stepS2004.

In step S2004, the data acquisition unit 1809 collates the subjectdetected in step S2001 with data of the disease for the subject, on theconnected network. For example, the data acquisition unit 1809 accessesan electronic chart database and collates the detected subject with themedical conditions or diagnosis result of the subject. Upon completionof the collation between the subject and the data, the processingproceeds to step S2005.

In step S2005, the data acquisition unit 1809 determines whether thedata for the detected subject is present as a result of the collationperformed in step S2004. When the data acquisition unit 1809 determinesthat the data for the detected subject is present (YES in step S2005),the processing proceeds to step S2006. In contrast, when the dataacquisition unit 1809 determines that the data for the detected subjectis absent (NO in step S2005), the processing proceeds to step S2007.

In step S2006, the data acquisition unit 1809 acquires the data for thedetected subject. Upon completion of the acquisition of the data for thedetected subject, the processing proceeds to step S2007.

In step S2007, the data acquisition unit 1809 determines whether thecontamination information resulting from the detected subject is presentin the contamination information stored in the contamination informationgeneration unit 107.

When the data acquisition unit 1809 determines that the contaminationinformation resulting from the detected subject is present (YES in stepS2007), the processing proceeds to step S2008. In contrast, when thedata acquisition unit 1809 determines that the contamination informationresulting from the detected subject is absent (NO in step S2007), theprocessing proceeds to step S2009.

In step S2008, the data acquisition unit 1809 acquires the contaminationinformation resulting from the detected subject. Upon completion of theacquisition of the contamination information resulting from the detectedsubject, the processing proceeds to step S2009.

In step S2009, the data acquisition unit 1809 stores each piece of theacquired information in an associated way. In this case, thecorrespondence between pieces of the stored information can berepresented in an associated way as in the table illustrated in FIG.21A. FIG. 21A illustrates data of the temperature information, medicalconditions, and contamination information for four different subjects (Ato D) stored in an associated way. Contaminated areas A, B, and C willbe described below.

The cleaning operation performed by the cleaning apparatus 1801 will nowbe described with reference to FIGS. 22 and 23 . FIG. 22 illustrates thecleaning operation performed by the cleaning apparatus 1801. FIG. 22illustrates a state where three different contaminated areas(contaminated areas A, B, and C) resulting from a person (notillustrated) are present in a room 2201 where the cleaning apparatus1801 is installed. Referring to FIG. 22 , configurations identical tothose of the cleaning apparatus 1801 illustrated in FIG. 18 are assignedthe same reference numerals, and descriptions thereof will be omitted.

A contaminated area 2202 indicates the shape of the contaminated areaincluded in the contamination information resulting from the subject A.The contaminated area 2202 is the contaminated area A in the tableillustrated in FIG. 21A. A spray range 2203 indicates the shape of thespray range of the cleaning agent performed on the contaminated area2202 by the first cleaning operation unit 1805.

A contaminated area 2204 indicates the shape of the contaminated areaincluded in the contamination information resulting from the subject C.The contaminated area 2204 is the contaminated area C in the tableillustrated in FIG. 21A. A spray range 2205 indicates the shape of thespray range of the cleaning agent performed on the contaminated area2204 by the first cleaning operation unit 1805.

A contaminated area 2206 indicates the shape of the contaminated areaincluded in the contamination information resulting from the subject B.The contaminated area 2206 is the contaminated area B in the tableillustrated in FIG. 21A. The contaminated area 2206 is placed at aposition below a stillage 2207 where the first cleaning operation unit1805 cannot spray the cleaning agent.

This completes the descriptions of the status of the room 2201 includingthe cleaning apparatus 1801 according to the present exemplaryembodiment.

FIG. 23 is a flowchart illustrating an example of the cleaning operationprocessing performed by the cleaning apparatus 1801. For example, afterthe cleaning apparatus 1801 completes the contaminated area detectionand generation processing illustrated in FIG. 19 , the cleaningapparatus 1801 automatically starts the cleaning operation processingillustrated in FIG. 23 when a predetermined time duration has elapsed.The cleaning apparatus 1801 may also start the cleaning operationprocessing illustrated in FIG. 23 upon reception of an instruction fromthe user.

When the cleaning operation processing is started, then in step S2301,the priority determination unit 1810 refers to the data stored in thedata acquisition unit 1809 and sets the priority for cleaning eachcontaminated area based on the body temperature information and themedical conditions of the subject. For example, the priorities when thedata illustrated in FIG. 21A is referred to are as illustrated in FIG.21B. These priorities are determined by a program developed based onspecialized experiences. For example, if a predetermined bodytemperature is exceeded, a higher priority is set to the contaminatedarea resulting from a subject with a higher body temperature. Inaddition, a higher priority is set to the contaminated area resultingfrom a subject in medical conditions with a higher risk of infection.Upon completion of the cleaning priority setting, the processingproceeds to step S2302.

In step S2302, the cleaning operation control unit 1804 determines theoperation contents of the first cleaning operation unit 1805 and thesecond cleaning operation unit 1806 based on the contaminationinformation and the priority set in step S2301.

Firstly, the cleaning operation control unit 1804 determines theoperation contents of the first cleaning operation unit 1805. In thecase of the example illustrated in FIG. 22 , the first cleaningoperation unit 1805 can clean the contaminated areas 2202 and 2204. Ahigher priority is set to the contaminated area 2204 (contaminated areaC) than to the contaminated area 2202 (contaminated area A). Thus, theoperation contents of the first cleaning operation unit 1805 include anoperation for spraying the cleaning agent to the contaminated area 2204for a first predetermined time duration. Upon completion of the sprayingof the cleaning agent to the contaminated area 2204, the first cleaningoperation unit 1805 performs an operation for spraying the cleaningagent to the contaminated area 2202 for a second predetermined timeduration which is shorter than the first predetermined time duration.

Then, the cleaning operation control unit 1804 determines the operationcontents of the second cleaning operation unit 1806. Referring to theexample illustrated in FIG. 22 , the second cleaning operation unit 1806can clean the contaminated areas 2204 and 2206. A higher priority is setto the contaminated area 2204 (contaminated area C) than to thecontaminated area 2206 (contaminated area B). However, since the firstcleaning operation unit 1805 firstly sprays the cleaning agent to thecontaminated area 2204, the operation contents of the second cleaningoperation unit 1806 includes an operation of irradiating thecontaminated area 2206 with an ultraviolet ray for a third predeterminedtime duration. When the first cleaning operation unit 1805 completes thespraying of the cleaning agent to the contaminated area 2204 aftercompletion of the ultraviolet radiation to the contaminated area 2206,the operation contents of the first cleaning operation unit 1805 includean operation for irradiating the contaminated area 2204 with anultraviolet ray for a fourth predetermined time duration which is longerthan the third predetermined time duration.

Upon completion of the determination of the operation contents of thefirst cleaning operation unit 1805 and the second cleaning operationunit 1806 in this way, the processing proceeds to step S2303.

In step S2303, the cleaning operation control unit 1804 controls thefirst cleaning operation unit 1805 and second cleaning operation unit1806 to perform cleaning based on the operation contents determined instep S2302. Upon completion of the control of the first cleaningoperation unit 1805 and the second cleaning operation unit 1806 based onthe operation contents, the processing proceeds to step S2304.

In step S2304, the cleaning operation control unit 1804 determineswhether the first cleaning operation unit 1805 and the second cleaningoperation unit 1806 have completed all of the operation contentsdetermined in step S2302. When the cleaning operation control unit 1804determines that not all of the operation contents are completed, i.e.,any of the operation contents is incomplete (NO in step S2304), theprocessing returns to step S2303. In step S2303, the cleaning operationcontrol unit 1804 continues the cleaning operation. In contrast, whenthe cleaning operation control unit 1804 determines that all of theoperation contents determined in step S2302 are completed (YES in stepS2304), the cleaning operation control unit 1804 completes the cleaningoperation processing.

By performing the above-described processing, the cleaning apparatus1801 can efficiently and automatically clean the areas contaminated bydroplets.

Cleaning result determination processing performed by the cleaningresult determination unit 1811 will now be described with reference toFIG. 24 . FIG. 24 is a flowchart illustrating an example of the cleaningresult determination processing.

When the cleaning result determination processing is started, then instep S2401, the cleaning result determination unit 1811 determines thecleaning result. The cleaning result determination unit 1811 determinesthe cleaning result based on the priority data illustrated in FIG. 21Band the cleaning operation contents of the first cleaning operation unit1805 and the second cleaning operation unit 1806.

For example, as illustrated in FIG. 21C, both the spraying of thecleaning agent and the ultraviolet irradiation are completed for thecontaminated area C having the “High” priority. Thus, the cleaningresult determination is successful. However, for example, when thepriority of the contaminated area B is also “High”, as illustrated inFIG. 21D, only the ultraviolet irradiation is completed for thecontaminated area B. Thus, the cleaning result determination is failed(manual cleaning required).

Upon completion of the cleaning result determination, the processingproceeds to step S2402.

In step S2402, the cleaning result determination unit 1811 determineswhether the cleaning result determination is successful. When thecleaning result determination unit 1811 determines that the cleaningresult determination is not successful (NO in step S2402), theprocessing proceeds to step S2403. In step S2403, the cleaning resultdetermination unit 1811 determines that the manual cleaning is required,notifies a predetermined contact, and then completes the cleaning resultdetermination processing. When the cleaning result determination unit1811 determines that the cleaning result determination is successful(YES in step S2402), the cleaning result determination unit 1811completes the cleaning result determination processing.

This processing brings efficient cleaning even in a case of aninsufficient cleaning result through mechanical cleaning. Assume anexample case where the cleaning result determination unit 1811 notifiesa predetermined contact, and a designated sanitation worker isdispatched to perform the cleaning. In this case, the informationprocessing apparatus 101 performs the contamination informationpresentation processing according to the first exemplary embodiment,enabling the sanitation worker to recognize the contaminated areas andefficiently perform the cleaning.

The first cleaning operation unit 1805 may be a cleaning agent nebulizercapable of selecting a type of cleaning agent from a plurality of typesof cleaning agents and then spraying the cleaning agent. In this case,the first cleaning operation unit 1805 may select the type of thecleaning agent to be sprayed, based on the data as illustrated in FIG.21B. This processing allows further improvement of the cleaning effectfor each contaminated area. When a plurality of cleaning operation unitscan perform the cleaning as is the case with the contaminated area 2204illustrated in FIG. 22 , there may be one or more cleaning operationunits that do not clean any contaminated area depending on the priority.This processing enables completing cleaning in a short time.

According to the sixth exemplary embodiment, the cleaning apparatus 1801sets the priority for each of the detected contaminated areas andselects a most suitable cleaning method, thus enabling efficientcleaning.

FIG. 25 is a block diagram illustrating an example configuration of aninformation processing apparatus 2501 according to a seventh exemplaryembodiment. Referring to FIG. 25 , components having identical functionsto those illustrated in FIG. 1 are assigned the same reference numeralsand duplicated descriptions thereof will be omitted. The informationprocessing apparatus 2501 includes a sound acquisition unit 102, a soundinformation generation unit 103, a sound direction detection unit 104,an imaging unit 105, a facial state detection unit 106, a contaminationinformation generation unit 107, an information presentation unit 108,and a communication bus 110. The information processing apparatus 2501also includes a system control unit 2502, a wireless communication unit2504, and a cleaning unit 2505.

The system control unit 2502 has a similar function to the systemcontrol unit 109 according to the first exemplary embodiment andcontrols the entire information processing apparatus 2501 via thecommunication bus 110. The system control unit 2502 also includes acleaning path generation unit 2503.

The cleaning path generation unit 2503 generates a cleaning path of thecleaning unit 2505 via the communication bus 110 by executing a programrecorded in a storage unit such as a nonvolatile memory (notillustrated). The cleaning path generation unit 2503 generates acleaning path to enable the cleaning unit 2505 to clean the detectedcontaminated areas in a shortest time or a shortest path. The cleaningpath generation unit 2503 generates a cleaning path so that thecontaminated areas are cleaned in a preferential way. When cleaning adetected contaminated area, cleaning on a one-way basis is recommended.This is because, in a case where an area is wiped and not contaminatedor a cleaned area is cleaned, contaminated areas may possibly beexpanded. One-way cleaning will be specifically described below withreference to the accompanying drawings.

The wireless communication unit 2504 includes an antenna for wirelesscommunication and a communication control unit. The wirelesscommunication unit 2504 wirelessly communicates with the cleaning unit2505 by using, for example, a Wireless Local Area Network (WirelessLAN). The information processing apparatus 2501 is capable of wirelesslycommunicating with electronic devices conforming to a wireless LAN. Thesystem control unit 2502 transmits cleaning path information generatedby the cleaning path generation unit 2503 to the cleaning unit 2505 viathe wireless communication unit 2504. The cleaning unit 2505 can alsotransmit information indicating the cleaning status to the systemcontrol unit 2502 through communication using the wireless LAN.

The cleaning unit 2505 performs cleaning based on the cleaning pathgenerated by the cleaning path generation unit 2503. The cleaning unit2505 is, for example, a self-propelled robot cleaner. Cleaning accordingto the seventh exemplary embodiment refers to wiping, for example, thefloor. The cleaning unit 2505 performs the cleaning by using a cleaningsheet having sanitization and/or sterilization effects for wiping. Thecleaning sheet may be suitably changeable. If the cleaning sheet isunchangeable, the cleaning unit 2505 may be provided with a mechanismfor sanitizing and/or sterilizing the cleaning sheet itself. Thecleaning unit 2505 can perform cleaning by using not only a cleaningsheet having sanitization and/or sterilization effects but also acleaning sheet coping with dust, food particles, smears, and otherstains on the floor. The cleaning unit 2505 can also perform thecleaning while changing the cleaning sheets as required. The cleaningunit 2505 is capable of lifting the cleaning sheet to avoid contact withthe floor and temporarily detaching the cleaning sheet. The cleaningunit 2505 can thereby be moved without staining the floor because thecleaning sheet does not come into contact with the floor.

The cleaning unit 2505 may operate in a plurality of operation modes.Examples of the operations modes include a contaminated area cleaningmode of cleaning a contaminated area by using a cleaning sheet havingsanitization and/or sterilization effects, and a normal cleaning mode ofcleaning the floor by using a cleaning sheet coping with dust, foodparticles, smears, and other stains. Examples of the operations modesalso include a moving mode in which the cleaning sheet is lifted ortemporarily detached to avoid contact with the floor. For switchingbetween these operation modes, the system control unit 2502 can controlthe operation mode switching via the wireless communication unit 2504,or the cleaning unit 2505 may determine the operation move switching.

The operation of the information processing apparatus 2501 will now bedescribed.

Firstly, the information processing apparatus 2501 generatescontaminated area detection and contamination information generationprocessing. The contaminated area detection and generation processingperformed by the information processing apparatus 2501 is similar to thecontaminated area detection and generation processing according to thefirst exemplary embodiment described above with reference to FIGS. 2 to4 , and thus descriptions thereof will be omitted.

The information processing apparatus 2501 generates a cleaning pathbased on the contamination information generated in the contaminatedarea detection and generation processing, and the cleaning unit 2505performs the cleaning operation based on the generated cleaning path.These pieces of processing will be described below with reference toFIGS. 26 to 28 . Pieces of the cleaning operation processing illustratedin FIGS. 26 to 28 may be independently executed or executed incombination as required.

FIG. 26 is a flowchart illustrating an example of the cleaning operationprocessing. For example, when a predetermined time duration has elapsedsince the information processing apparatus 2501 completes thecontaminated area detection and generation processing, the informationprocessing apparatus 2501 automatically starts the cleaning operationprocessing illustrated in FIG. 26 . The information processing apparatus2501 may also start the cleaning operation processing illustrated inFIG. 26 upon reception of an instruction from the user.

When the cleaning operation processing is started, then in step S2601,the cleaning path generation unit 2503 generates a cleaning path of thecleaning unit 2505 based on the contamination information generated bythe contamination information generation unit 107. The cleaning pathgenerated in this case is a path for cleaning each contaminated area ona one-way basis. The cleaning path generation unit 2503 determines apath such that the cleaning unit 2505 does not wipe any area having beenonce cleaned, by using the same cleaning sheet. Upon completion of thecleaning path generation, the processing proceeds to step S2602.

In step S2602, the system control unit 2502 transmits the cleaning pathinformation generated in step S2601 to the cleaning unit 2505 via thewireless communication unit 2504. Upon completion of the cleaning pathinformation transmission to the cleaning unit 2505, the processingproceeds to step S2603.

In step S2603, the cleaning unit 2505 starts cleaning based on thecleaning path information received in step S2602. Then, the processingproceeds to step S2604.

In step S2604, the cleaning unit 2505 determines whether a predeterminedtime duration has elapsed since the cleaning unit 2505 started thecleaning. When the cleaning unit 2505 determines that the predeterminedtime duration has elapsed since the cleaning unit 2505 started thecleaning (YES in step S2604), the processing proceeds to step S2606. Incontrast, when the cleaning unit 2505 determines that the predeterminedtime duration has not elapsed since the cleaning unit 2505 started thecleaning (NO in step S2604), the processing proceeds to step S2605.

In step S2605, the cleaning unit 2505 recognizes the total cleaned aresince the cleaning unit 2505 started the cleaning to determine whetherthe total cleaned area is larger than or equal to a predetermined area.When the cleaning unit 2505 determines that the total cleaned area islarger than or equal to the predetermined area (YES in step S2605), theprocessing proceeds to step S2606. When the cleaning unit 2505determines that the total cleaned area is less than the predeterminedarea (NO in step S2605), the processing returns to step S2603.

In step S2606, the cleaning unit 2505 changes the cleaning sheet orsanitizes the cleaning sheet currently being used for cleaning. Uponcompletion of the cleaning sheet change or sanitization, the processingproceeds to step S2607.

In step S2607, the cleaning unit 2505 determines whether all of thecontaminated areas in the cleaning path information received in stepS2602 have been cleaned. When the cleaning unit 2505 determines that anypart of the contaminated areas remains not cleaned (NO in step S2607),the processing returns to step S2603. In contrast, when the cleaningunit 2505 determines that all of the contaminated areas have beencleaned (YES in step S2607), the cleaning unit 2505 completes thecleaning operation processing.

The above-descried processing enables the cleaning unit 2505 to cleaneach contaminated area on a one-way basis without expanding thecontaminated area. The contaminated areas can be accurately cleaned bychanging and/or sanitizing the cleaning sheet as required.

FIG. 27 is a flowchart illustrating an example of the cleaning operationprocessing. For example, when a predetermined time duration has elapsedsince the information processing apparatus 2501 completes thecontaminated area detection and generation processing, the informationprocessing apparatus 2501 automatically starts the cleaning operationprocessing illustrated in FIG. 27 . The information processing apparatus2501 may also start the cleaning operation processing illustrated inFIG. 27 upon reception of an instruction from the user.

When the cleaning operation processing is started, then in step S2701,the cleaning path generation unit 2503 generates a cleaning path of thecleaning unit 2505 based on the contamination information generated bythe contamination information generation unit 107. The cleaning pathgenerated in this case is a path for cleaning each contaminated area ona one-way basis. The cleaning path generation unit 2503 determines apath such that the cleaning unit 2505 does not wipe any area having beenonce cleaned, by using the same cleaning sheet. Upon completion of thecleaning path generation, the processing proceeds to step S2702.

In step S2702, the system control unit 2502 transmits the cleaning pathinformation generated in step S2701 to the cleaning unit 2505 via thewireless communication unit 2504. Upon completion of the cleaning pathinformation transmission to the cleaning unit 2505, the processingproceeds to step S2703.

In step S2703, the cleaning unit 2505 starts cleaning based on thecleaning path information received in step S2702. Then, the processingproceeds to step S2704.

In step S2704, the cleaning unit 2505 sets the area cleaned in stepS2703 as an entry restricted area where the cleaning unit 2505 cannotenter. The cleaning unit 2505 may store entry restricted area settinginformation in a recording medium (not illustrated). The cleaning unit2505 may transmit the entry restricted area setting information to theinformation processing apparatus 2501 via the wireless communicationunit 2504. The cleaning path generation unit 2503 may generate acleaning path again based on the received entry restricted area settinginformation and then transmit the information to the cleaning unit 2505.Upon completion of the entry restricted area setting, the processingproceeds to step S2705.

In step S2705, the cleaning unit 2505 determines whether the cleaningsheet currently being used for cleaning is changed and sanitized or islifted and removed. When the cleaning unit 2505 determines that thecleaning sheet currently being used for cleaning is changed andsanitized or is lifted and removed (YES in step S2705), the processingproceeds to step S2706. When the cleaning unit 2505 determines that thecleaning sheet is neither changed and sanitized nor is lifted andremoved (NO in step S2705), the processing returns to step S2703.

In step S2706, the cleaning unit 2505 cancels the entry restricted areaset in step S2704. The cleaning unit 2505 may store entry restrictedarea cancel information in a recording medium (not illustrated). Thecleaning unit 2505 may transmit the entry restricted area cancelinformation to the information processing apparatus 2501 via thewireless communication unit 2504. The cleaning path generation unit 2503may generate a cleaning path again based on the received entryrestricted area cancel information and then transmit the information tothe cleaning unit 2505. Upon completion of the entry restricted areacancellation, the processing proceeds to step S2707.

In step S2707, the cleaning unit 2505 determines whether all of thecontaminated areas included in the cleaning path information receivedfrom the system control unit 2502 have been cleaned. When the cleaningunit 2505 determines that any of the contaminated areas remains notcleaned (NO in step S2707), the processing returns to step S2703. Incontrast, when the cleaning unit 2505 determines that all of thecontaminated areas have been cleaned (YES in step S2707), the cleaningunit 2505 completes the cleaning operation processing.

Performing processing in this way enables preventing the contaminatedareas from being expanded when the cleaning unit 2505 keeps cleaning thecontaminated areas in a state where viruses adhere to the cleaning unit2505.

In a case where a plurality of the cleaning units 2505 is provided, theinformation processing apparatus 2501 acquires, from each cleaning unit2505, information about the areas cleaned by each cleaning unit 2505,and the cleaning path generation unit 2503 updates the cleaning path.This enables generating a cleaning path such that each cleaning unitcontaminates no cleaned region.

FIG. 28 is a flowchart illustrating an example of the cleaning operationprocessing. For example, when a predetermined time duration has elapsedsince the information processing apparatus 2501 completes thecontaminated area detection and generation processing, the informationprocessing apparatus 2501 automatically starts the cleaning operationprocessing illustrated in FIG. 28 . The information processing apparatus2501 may also start the cleaning operation processing illustrated inFIG. 28 upon reception of an instruction from the user.

When the cleaning operation processing is started, then in step S2801,the cleaning path generation unit 2503 generates a cleaning path of thecleaning unit 2505 based on the contamination information generated inthe contamination information generation unit 107. The cleaning pathgenerated in this case is a path for cleaning each contaminated area ona one-way basis. The cleaning path generation unit 2503 determines apath such that the cleaning unit 2505 does not wipe any area having beenonce cleaned, by using the same cleaning sheet. Upon completion of thecleaning path generation, the processing proceeds to step S2802.

In step S2802, the system control unit 2502 transmits the cleaning pathinformation generated in step S2801 to the cleaning unit 2505 via thewireless communication unit 2504. Upon completion of the cleaning pathinformation transmission to the cleaning unit 2505, the processingproceeds to step S2803.

In step S2803, the system control unit 2502 transmits an operation modechange request to the cleaning unit 2505 via the wireless communicationunit 2504 to change the operation mode to the contaminated area cleaningmode. Upon completion of the transmission of the operation mode changerequest, the processing proceeds to step S2804.

In step S2804, the cleaning unit 2505 changes the operation mode to thecontaminated area cleaning mode according to the operation mode changerequest received in step S2803. After the operation mode is changed, thecleaning unit 2505 starts cleaning based on the cleaning pathinformation received in step S2802. Then, the processing proceeds tostep S2805.

In step S2805, the system control unit 2502 determines whether cleaningstatus information for the cleaning unit 2505 is received via thewireless communication unit 2504. When the system control unit 2502determines that the cleaning status information for the cleaning unit2505 is receive (YES in step S2805), the processing proceeds to stepS2806. In contrast, when the system control unit 2502 determines that nocleaning status information for the cleaning unit 2505 is received (NOin step S2805), the system control unit 2502 waits for the reception ofthe cleaning status information. The cleaning status informationincludes information that indicates the contaminated areas that remainnot cleaned during the cleaning performed by the cleaning unit 2505based on the cleaning path information.

In step S2806, the cleaning path generation unit 2503 regenerates acleaning path of the cleaning unit 2505 based on the cleaning pathinformation generated in step S2801 and the cleaning status informationreceived in step S2805. According to the regenerated cleaning pathinformation, a path is set to clean the areas again which remained notcleaned based on the cleaning path information generated is step S2801.After completion of the processing in step S2802, the cleaning pathgeneration unit 2503 may regenerate a cleaning path including newcontaminated areas. Upon completion of the cleaning path regeneration,the processing proceeds to step S2807.

In step S2807, the system control unit 2502 transmits the cleaning pathinformation generated in step S2806 to the cleaning unit 2505 via thewireless communication unit 2504. Upon completion of the transmission ofthe regenerated cleaning path information to the cleaning unit 2505, theprocessing proceeds to step S2808.

In step S2808, the cleaning unit 2505 starts cleaning based on theregenerated cleaning path information received in step S2807. Then, theprocessing proceeds to step S2809.

In step S2809, the cleaning unit 2505 determines whether all of thecontaminated areas included in the cleaning path information receivedfrom the system control unit 2502 have been cleaned. When the cleaningunit 2505 determines that any of the contaminated areas remains notcleaned (NO in step S2809), the processing returns to step S2808. Incontrast, when the cleaning unit 2505 determines that all of thecontaminated areas have been cleaned (YES in step S2809), the cleaningunit 2505 completes the cleaning operation processing.

Performing processing in this way enables generating a cleaning path inview of the actual cleaning status of the cleaning unit 2505 andtherefore accurately performing the cleaning processing.

An example of the cleaning processing performed by the cleaning unit2505 will now be described with reference to FIGS. 29A and 29B.

FIG. 29A illustrates an example of the cleaning operation of thecleaning unit 2505 in a room 2901 where the information processingapparatus 2501 is installed. Contaminated areas 2902, 2903 and 2904indicate the shapes of contaminated areas included in the contaminationinformation generated in the contaminated area detection and generationprocessing and then stored.

Cleaning paths 2905 a, 2505 b and 2905 c drawn with broken lines arecleaning paths when the cleaning unit 2505 is operating in the normalcleaning mode or the moving mode. Cleaning paths 2906 a, 2906 b and 2906c drawn with solid lines are cleaning paths used when the cleaning unit2505 is operating in the contaminated area cleaning mode.

A cleaning method performed by the cleaning unit 2505 to clean the room2901 based on the cleaning path information received from the systemcontrol unit 2502 will now be specifically described.

Firstly, in the cleaning path 2905 a, the cleaning unit 2505 isoperating in the normal cleaning mode or the moving mode. When thecleaning unit 2505 is operating in the normal cleaning mode before thereception of the cleaning path information, the cleaning unit 2505 maycontinue the operation in the normal cleaning mode until the cleaningunit 2505 reaches the contaminated area 2902. When the cleaning unit2505 reaches the contaminated area 2902, the cleaning unit 2505 maychange the operation mode to the contaminated area cleaning modeaccording to the request of the information processing apparatus 2501.

The cleaning unit 2505 operates in the contaminated area cleaning modealong the cleaning path 2906 a. In this case, the cleaning unit 2505cleans the contaminated area 2902 on a one-way basis. When apredetermined time duration has elapsed during the cleaning of thecontaminated area 2902 or when the total cleaned area reaches thepredetermined area, the cleaning sheet is changed or sanitized asrequired. When the cleaning of the contaminated area 2902 is completed,the cleaning unit 2505 changes the operation mode to the normal cleaningmode or the moving mode. The cleaning unit 2505 sets the contaminatedarea 2902 having been cleaned as an entry restricted area.Alternatively, the cleaning unit 2505 may transmit the cleaning statusinformation indicating that the contaminated area 2902 is set as anentry restricted area to the information processing apparatus 2501 viathe wireless communication unit 2504.

Along the cleaning path 2905 b, the cleaning unit 2505 operates in thenormal cleaning mode or the moving mode again. When performing thecleaning in the normal cleaning mode, the cleaning unit 2505 operates toclean only contaminations on the shortest path to the contaminated area2903. The cleaning unit 2505 may transmit the cleaning statusinformation indicating that the cleaning of the cleaning path 2905 b iscompleted or incomplete to the information processing apparatus 2501 viathe wireless communication unit 2504. When the cleaning unit 2505reaches the contaminated area 2903, the cleaning unit 2505 changes theoperation mode to the contaminated area cleaning mode according to therequest of the information processing apparatus 2501.

Along the cleaning path 2906 b, the cleaning unit 2505 operates in thecontaminated area cleaning mode. In this case, the cleaning unit 2505cleans the contaminated area 2903 on a one-way basis. The cleaning unit2505 performs similar processing to that for the cleaning path 2906 a,and detailed descriptions thereof will be omitted.

Along the cleaning path 2905 c, the cleaning unit 2505 changes theoperation mode to the normal cleaning mode or the moving mode againbefore the operation. The cleaning unit 2505 performs similar processingto that for the cleaning paths 2905 a and 2905 b, and detaileddescriptions thereof will be omitted.

Along the cleaning path 2906 c, the cleaning unit 2505 operates in thecontaminated area cleaning mode. In this case, the cleaning unit 2505cleans the contaminated area 2904 on a one-way basis. The cleaning unit2505 performs similar processing to that for the cleaning paths 2906 aand 2906 b, and detailed descriptions thereof will be omitted.

When any of the cleaning paths 2905 a, 2905 b, and 2905 c remains notcleaned after completion of the cleaning of the contaminated area 2904,the cleaning unit 2505 may advance to an unclean position in the movingmode and then continue cleaning in the normal cleaning mode.

FIGS. 29B and 29C illustrate examples of the cleaning paths generated bythe cleaning path generation unit 2503 using arrows. The cleaning unit2505 cleans the contaminated area 2902 while moving along the arrow inthe contaminated area 2902. Referring to the example illustrated in FIG.29B, the cleaning unit 2505 starts the cleaning from a start point S,repeats the sequence of reaching an edge of the contaminated area 2902and turning down, and reaches an end point E on a one-way basis. In thiscase, the cleaning unit 2505 does not return on the cleaned path. Thecleaning unit 2505 may change or sanitize the cleaning sheet in themiddle of the arrow as required.

Referring to the example illustrated in FIG. 29C, the cleaning unit 2505starts the cleaning from a start point S1 and reaches an end point E1placed at an edge of the contaminated area 2902 on a one-way basis. Inthis case, the cleaning unit 2505 does not return on the cleaned path.The cleaning unit 2505 moves from the end point E1 to a start point S2which is the start point for the next cleaning operation. In this case,the cleaning unit 2505 moves in the moving mode. The cleaning unit 2505may change or sanitize the cleaning sheet during the movement from eachend point to the next start point, or change or sanitize the cleaningsheet in the middle of the arrow as required.

According to the seventh exemplary embodiment, the cleaning unit 2505can efficiently and accurately clean the detected contaminated area.

Some embodiments can also be achieved when a program for implementing atleast one of the functions according to the above-described exemplaryembodiments is supplied to a system or apparatus via a network orstorage medium, and at least one processor in a computer of the systemor apparatus reads and executes the program. Further, some embodimentscan also be achieved by a circuit (e.g., an application specificintegrated circuit (ASIC)) for implementing at least one function.

The above-described exemplary embodiments are to be merely considered asillustrative in embodying the present disclosure, and are not to beinterpreted as restrictive on the technical scope of the presentdisclosure. More specifically, various embodiments may be embodied indiverse forms without departing from the technical concepts or essentialcharacteristics thereof.

Embodiments of the present disclosure make it possible to identify acontaminated area to which droplets scattered from a person adhere, andeasily recognize the area to be cleaned.

Other Embodiments

Some embodiment(s) can also be realized by a computer of a system orapparatus that reads out and executes computer-executable instructions(e.g., one or more programs) recorded on a storage medium (which mayalso be referred to more fully as a ‘non-transitory computer-readablestorage medium’) to perform the functions of one or more of theabove-described embodiment(s) and/or that includes one or more circuits(e.g., application specific integrated circuit (ASIC)) for performingthe functions of one or more of the above-described embodiment(s), andby a method performed by the computer of the system or apparatus by, forexample, reading out and executing the computer-executable instructionsfrom the storage medium to perform the functions of one or more of theabove-described embodiment(s) and/or controlling the one or morecircuits to perform the functions of one or more of the above-describedembodiment(s). The computer may comprise one or more processors (e.g.,central processing unit (CPU), micro processing unit (MPU)) and mayinclude a network of separate computers or separate processors to readout and execute the computer-executable instructions. Thecomputer-executable instructions may be provided to the computer, forexample, from a network or the storage medium. The storage medium mayinclude, for example, one or more of a hard disk, a random-access memory(RAM), a read only memory (ROM), a storage of distributed computingsystems, an optical disk (such as a compact disc (CD), digital versatiledisc (DVD), or Blu-ray Disc (BD)™), a flash memory device, a memorycard, and the like.

While the present disclosure has described exemplary embodiments, it isto be understood that some embodiments are not limited to the disclosedexemplary embodiments. The scope of the following claims is to beaccorded the broadest interpretation so as to encompass all suchmodifications and similar structures and functions.

This application claims priority to Japanese Patent Application No.2021-146854, which was filed on Sep. 9, 2021 and which is herebyincorporated by reference herein in its entirety.

What is claimed is:
 1. An information processing apparatus comprising: asound information generation unit configured to generate soundinformation in which a physiological sound of a body is extracted froman acquired sound; a sound direction detection unit configured to detecta direction of the physiological sound of the body based on the soundinformation; a facial state detection unit configured to detect a stateof a face existing in the direction of the physiological sound of thebody detected by the sound direction detection unit, from an imagecaptured and generated by an imaging unit; and a contaminationinformation generation unit configured to generate contaminationinformation in which a predetermined area according to the state of theface is set as a contaminated area, based on the state of the facedetected by the facial state detection unit.
 2. The informationprocessing apparatus according to claim 1, wherein the contaminationinformation includes at least one of a distance, a direction, and ashape of contamination.
 3. The information processing apparatusaccording to claim 1, wherein the state of the face detected by thefacial state detection unit includes a face orientation, and wherein thecontamination information generation unit generates the contaminationinformation in which a predetermined area in a front direction of theface detected by the facial state detection unit is set as acontaminated area.
 4. The information processing apparatus according toclaim 1, wherein the state of the face detected by the facial statedetection unit includes a mask wearing state, and wherein, in a casewhere the state of the face detected by the facial state detection unitis a face wearing a mask, the contamination information generation unitchanges the contaminated area in the contamination information.
 5. Theinformation processing apparatus according to claim 1, wherein the soundinformation generation unit generates sound volume informationindicating the sound volume of the physiological sound of the body, andwherein the contamination information generation unit changes thecontaminated area in the contamination information based on the soundvolume information generated by the sound information generation unit.6. The information processing apparatus according to claim 1, whereinthe state of the face detected by the facial state detection unitincludes a distance between the imaging unit and the face, and whereinthe contamination information includes a position of a contaminated areabased on the distance between the imaging unit and the face detected bythe facial state detection unit and the direction of the physiologicalsound of the body detected by the sound direction detection unit.
 7. Theinformation processing apparatus according to claim 1, wherein thephysiological sound of the body includes at least a sound of anutterance, a sneeze, a cough, a blow, and a exhalation.
 8. Theinformation processing apparatus according to claim 1, furthercomprising an information presentation unit configured to present thegenerated contamination information.
 9. The information processingapparatus according to claim 8, wherein the information presentationunit converts the contamination information into at least either one ofa sound and a video.
 10. The information processing apparatus accordingto claim 8, wherein the information presentation unit converts thecontamination information into a video, converts the video into mappinginformation in which the video is mapped on an object, and presents themapping information.
 11. The information processing apparatus accordingto claim 8, wherein the contamination information generation unitcombines pieces of the contamination information to generate combinedcontamination information, and wherein the information presentation unitpresents the combined contamination information generated by thecontamination information generation unit.
 12. The informationprocessing apparatus according to claim 11, wherein the contaminationinformation generation unit stores the contamination informationsatisfying a predetermined storage condition, combines pieces of thestored contamination information to generate the combined contaminationinformation, and deletes the contamination information satisfying apredetermined deletion condition out of the stored contaminationinformation.
 13. The information processing apparatus according to claim1, further comprising an object movement detection unit configured todetect a moving object in an image generated by the imaging unit,wherein the contamination information generation unit updates thecontamination information corresponding to a movement of the movingobject detected by the object movement detection unit.
 14. Theinformation processing apparatus according to claim 13, wherein theobject movement detection unit detects movement information including anarea where the moving object has existed before the movement and an areawhere the moving object exists after the movement, and wherein thecontamination information generation unit updates the contaminationinformation based on the movement information detected by the objectmovement detection unit.
 15. The information processing apparatusaccording to claim 14, wherein the movement information includes a locusrange where the moving object has moved.
 16. The information processingapparatus according to claim 1, further comprising a sound acquisitionunit configured to acquire a sound, wherein the sound informationgeneration unit generates the sound information based on the soundacquired by the sound acquisition unit.
 17. The information processingapparatus according to claim 1, further comprising the imaging unitconfigured to capture a subject to generate an image of the subject. 18.The information processing apparatus according to claim 1, furthercomprising a control unit configured to control a cleaning unit toperform cleaning of a contaminated area based on the contaminationinformation generated by the contamination information generation unit.19. The information processing apparatus according to claim 18, whereinthe control unit determines operation contents of the cleaning unitbased on the contamination information generated by the contaminationinformation generation unit and controls an operation of the cleaningunit.
 20. The information processing apparatus according to claim 18,further comprising: a data acquisition unit configured to acquire datafor a subject captured by the imaging unit; and a priority determinationunit configured to determine a priority of cleaning for a contaminatedarea based on data acquired by the data acquisition unit, wherein thecontrol unit controls the cleaning unit based on the priority ofcleaning and the contamination information generated by thecontamination information generation unit.
 21. A method for controllingan information processing apparatus, the method comprising: generatingsound information in which a physiological sound of a body is extractedfrom an acquired sound; detecting a direction of the physiological soundof the body based on the sound information; detecting a state of a faceexisting in the direction of the physiological sound of the bodydetected in the sound direction detection, from an image captured andgenerated by an imaging unit; and generating contamination informationin which a predetermined area according to the state of the face is setas a contaminated area, based on the state of the face.
 22. Anon-transitory computer-readable recording medium storing a program forcausing a computer to execute a method comprising: generating soundinformation in which a physiological sound of a body is extracted froman acquired sound; detecting a direction of the physiological sound ofthe body based on the sound information; detecting a state of a faceexisting in the direction of the physiological sound of the bodydetected in the sound direction detection, from an image captured andgenerated by an imaging unit; and generating contamination informationin which a predetermined area according to the state of the face is setas a contaminated area, based on the state of the face.